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Background / Context: 

The need for a seientifically-literate Ameriean population has gained great prominenee on the 
educational landscape of priorities over the past decade; young children will be increasingly 
exposed to the STEM fields and encouraged to excel in these areas (The White House, 2009). 
While experience and research suggest that teachers’ science knowledge is predictive of 
children’s science learning (Loucks-Horsley, Love, Stiles, Mundry, & Hewson, 2003), many 
early childhood teachers are not ready to engage children in rich science experiences that lay the 
groundwork for later success. Instead, teachers often rely on “spur of the moment” planning or 
provide activities that are loosely connected to a theme (Bowman, Donovan, & Burns, 2001). 
Even quality programs tend to emphasize language and social development, at the expense of 
science learning (Smith & Dickinson, 1994). These challenges are exacerbated by some teachers’ 
apprehension towards teaching science based on uncertainty of both the science content and 
pedagogy. When science instruction does occur, it usually focuses either on the recall of facts 
with insufficient attention to student understanding or, alternatively, on activities that are only 
loosely related to conceptual goals. Neither approach promotes teacher-student interaction that 
furthers science learning (Duschl, Schweingruber, & Shouse, 2007). 

Professional development is key to assuring that early childhood teachers provide children 
with cognitively-challenging early learning experiences (Bowman, et ah, 2001; Dwyer, Chait, & 
McKee, 2000; Espinosa, 2002; Helbum & Bergmann, 2002). Elnfortunately, few models of 
professional development build preschool teachers’ skills and knowledge in an ongoing way and 
provide access to content knowledge (Barnett, 2003; Whitebook, 2003). Recent research 
comparing different types of professional development for science teaching suggests that the 
most effective models are a hybrid, combining professional development with curriculum in 
ways that lead to intentional and informed use of curricular materials (Penuel & Gallagher, 

2008). In addition, effective teacher professional development should not only focus on 
developing teachers’ science content knowledge, but also their pedagogical content knowledge 
related to children’s early science development (Ball, 2000; Shulman, 1987). 

Purpose / Objective / Research Question / Focus of Study: 

We are responding to the critical need for empirical evidence on effective strategies to improve 
science instruction in preschool. Eocusing on the Head Start community. Foundations of Science 
Literacy (FSL) is a credit-bearing professional development course that directly addresses the 
achievement gap in early science education. The program not only addresses an urgent need, it 
also integrates the resources, structure, and support that preschool teachers need to improve early 
science learning and teaching. Based on many years of experience, we have learned that episodic 
workshops, offered without a sound curriculum or credit, do little to change teachers’ classroom 
practice. In sharp contrast, the great promise of FSL is that it includes several features that create 
a comprehensive approach. One new feature, under current development, is the use of embedded 
formative assessments to guide children’s science learning. In principle, formative assessment is 
a powerful tool for helping teachers understand children’s current knowledge and support their 
reasoning, reflection, and conceptual development. 

In this paper, we draw on two current lES-funded projects in order to address two research 
questions. Eirst, based on data from an efficacy study nearing completion, we assess whether 
FSL without embedded formative assessments leads to gains in preschool children’s 
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understanding of science and scientific thinking as mediated by improvement of preschool 
teachers’ pedagogical content knowledge and quality science instruction within the classroom. 
Second, drawing on insights from a new development project, we describe how instruction- 
embedded formative assessment can be used to strengthen and clarify the relation between gains 
in teacher pedagogical content knowledge and gains in children’s scientific thinking. In 
particular, we argue that formative assessment is critical in strengthening this relation, as it 
provides much-needed structure for preschool teachers to consolidate their pedagogical content 
knowledge in science, and translate it into effective instruction. In addition, unlike one-shot 
summative assessment, formative assessment is also critical in clarifying this relation, as it 
provides a means of tracking and relating the co-occurring changes in teachers’ instruction and 
children’s scientific thinking at various points during the implementation of the program. 

Setting: 

For the efficacy study, FSL was implemented in the greater New York City area. Implementation 
and research for Cohort 1 of the study were conducted during the 2009-2010 school year. (As 
available, we will also discuss data for Cohort 2, conducted during the 2011-2012 school year.) 
For the development project, FSL is being implemented in Hartford CT. 

Population / Participants / Subjects: 

Our population includes preschool teachers and a sample of children in their classrooms. The 
Cohort 1 analytic sample in the efficacy study is comprised of 40 intervention classrooms and 32 
control classrooms, with 436 children (270 in intervention classrooms and 186 in control). The 
teachers and children participating in the development project are drawn from three early 
childhood groups in Hartford: Community Renewal Teach Head Start, Harford Public Schools, 
and the Office for Young Children that oversees Hartford’s School Readiness Program. 

Intervention / Program / Practice: 

FSL as assessed in the efficacy study consists of three components: 1) instructional, face-to-face 
sessions that build teachers’ pedagogical content knowledge in physical science; 2) a coaching 
component that supports teachers as they master content and methods of inquiry in physical 
science; and 3) a unit of the Young Scientist Series, a unique preschool science curriculum for 4- 
year-olds in widespread use and with recognition from national science education organizations. 
In the development project, FSL is being expanded to include additional science domains (life 
and earth sciences; engineering) as well as a toolkit of formative assessments designed to help 
teachers assess children’s scientific understanding and thinking and plan effective instruction. 

Research Design (Efficacy Study): 

We used a randomized controlled trial (RCT) design with a total sample of 78 preschool 
classrooms, with 40 classrooms assigned to the intervention group and 32 classrooms assigned to 
the control group. During randomization, classrooms were blocked by program location (one of 
two locations) and by center. Children were then selected within each classroom. If we received 
consent forms for more than 10 eligible children within a classroom, the study team randomly 
selected 10 to be in the main sample. 

Data Collection and Analysis (Efficacy Study): 

Measures. Children were assessed one-on-one before and after the implementation of the FSL 
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professional development eourse, in the fall (Oct.-Nov.) and the spring (Apr.-Jun.). In this paper, 
we diseuss findings from predietion trials on the Preschool Assessment of Science {PAS), a 
measure of presehoolers’ knowledge and inquiry skills in physieal seienee. Predietion tasks 
measure ehildren’s ability to aecurately prediet a physieal event based on eritical observation of 
related events. Internal eonsisteney using Cronbaeh’s alpha (a) is 0.727. 

Analyses. Logistie regression analyses were eondueted in order to determine the effeet of FSL on 
speeifie items in the “Floating & Sinking”, “Marbles & Ramps”, and “Water Level” predietion 
tasks. Predietion tasks included three basic items: children’s predictions of a physical 
phenomenon before it occurs (initial prediction), their ability to accurately observe that physical 
phenomenon as it occurs (observation), and finally their ability to revise or confirm their initial 
prediction based on their observation in order to make a related prediction (revised prediction). 
Because these items are binomial (either correct or incorrect), logistic regression was used to 
estimate the change in probability of correct response associated with predictor variables. 

Findings / Implications: 

Findings. In order to evaluate the effect of the curriculum on children’s responses, children’s 
responses in the spring were treated as outcomes, while controlling for their responses in the fall. 
Models were built incrementally with blocks of related variables, beginning with child-level 
covariates that were hypothesized to contribute to performance (age, gender, language status, 
lEP status, maternal education, fall score on outcome variable, and cognitive flexibility), 
followed by the classroom-level variable of intervention condition, and finally, interaction terms 
between participation in FSL and each child-level covariate. Individual predictors and interaction 
terms were evaluated using the Wald Chi-square statistic of regression coefficients (Peng 2002). 
Interaction terms were excluded from the final model if the Wald test was non-significant at p of 
.10 or more. Because the study design was nested, with children nested in classrooms, standard 
logistic regression may underestimate standard errors and overestimate effects. To account for 
this possibility, hierarchical linear modeling (HLM) with Bernoulli estimation for binomial 
outcomes was used to conduct confirmatory tests of final models obtained using logistic 
regression. Significant associations are presented for both logistic regression and HLM analyses. 

In the “Floating & Sinking” task, children who participated in FSL were significantly more 
likely to make correct initial predictions, observations, and revised predictions. For the initial 
prediction responses, B = 1.03(0.45), Wald chi-square = 5.34, p < .05, controlling for all 
covariates. There were significant negative interaction effects between participation in FSL and 
gender, B = -1 .01(0.50), Wald chi-square = 4.08, p < .05, and between participation in FSL and 
maternal education, B = -1.46(0.64), Wald chi-square = 5.\9,p < .05. These interactions 
indicated that girls and children with mid-level maternal education (high school diploma and/or 
some college) who participated in FSL had a greater increase in the likelihood of correct initial 
prediction from fall to spring compared to boys and children with low maternal education (less 
than a high school diploma) in FSL. These results were confirmed by HLM analyses. For the 
observation responses, B = 0.52(0.25), Wald chi-square = 4.29, p < .05, controlling for all 
covariates, although the effect was very small. There were no significant interaction effects 
associated with participation in FSL. These results were confirmed by HLM analyses. The effect 
of FSL was only marginally significant according to HLM analyses, B = 0.51(0.31), t{6\) = 1.67, 
p = .10. For the revised predictions, B = 0.81(0.32), Wald chi-square = 6.31, p < .05, controlling 
for all covariates. There were no significant interaction effects associated with participation in 
FSL. These results were confirmed by HLM analyses. 
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In contrast, we found mixed results on the other two tasks. In the “Water Level” task, 
ehildren who partieipated in FSL were signifieantly more likely to make eorreet initial 
predietions and observations, but not revised predietions. For the initial predietions, B = 
0.48(0.24), Wald chi-square = 3.94, p < .05, eontrolling for all eovariates, although this effeet 
was very small. There were no statistieally signifieant interaction effects associated with 
partieipation in FSL. The effeet of FSL was only marginally signifieant aeeording to HLM 
analyses, B = 0.45(0.25), t(61) = 1.82, p = .073. For the observations, B = 1.01(0.37), Wald chi- 
square = 1 .IS, p < .01, eontrolling for all eovariates. There were signifieant negative interaetion 
effeets between partieipation in FSL and low maternal edueation, B = -1.72(0.67), Wald chi- 
square = 6.59, p < .05, and between partieipation in FSL and high maternal edueation, B = - 
1.44(0.60), Wald chi-square = 5.90, p < .05. These interaetions indieated that children with mid- 
level maternal edueation (high sehool diploma and/or some eollege) who partieipated in FSL had 
a greater increase in the likelihood of eorreet response from fall to spring eompared to ehildren 
with low maternal edueation (less than a high sehool diploma) or high maternal edueation 
(baehelor’s degree) in the intervention. Finally, for the “Marbles & Ramps” task, we found no 
signifieant group differences on any of the items based on partieipation in FSL. 

Implications. Although several items of this task showed evidenee of an effect of FSL, several 
items did not. These eonflieting results may be explained by 1) a laek of sensitivity of the 
measure to detect variation in children’s knowledge and inquiry skills and/or 2) an insuffieient 
strength of the intervention to ehange children’s knowledge and inquiry skills. In order to ensure 
reliability of administration for the purpose of summative assessment, this measure was 
intentionally designed to eonstrain the manner in whieh ehildren eould respond (e.g. answering 
elosed-ended questions, ehoosing one objeet among an array of objeets). This assessment format, 
then, also eonstrains the degree to whieh children ean demonstrate their learning, whieh may 
manifest itself through varied forms of expression (e.g. in conversation, in drawings, and in 
behavior). In eontrast, instruetion-embedded formative assessments, currently being piloted, will 
allow teaehers to assess ehildren’ s learning within the elassroom eontext in a way that is both 
more authentie and more flexible than summative assessment teehniques. In particular, through 
the use of “evidenee forms,” teaehers may prompt ehildren to make observations or predietions 
about a physieal phenomenon (e.g. what water will look like in a tube when it is tilted; how 
bloeks used for building structures may differ in shape or material). (Please insert Figure 1 here.) 
Children who are more verbal may respond by describing their prediction; others may draw a 
pieture; still others may pantomime. By probing further, teaehers ean gain greater insight into 
ehildren ’s learning by eneouraging them, not only to demonstrate their knowledge but, to 
demonstrate their ability to answer questions through active manipulation of materials (e.g. 
putting water in the tube and seeing what happens). 

Conclusions: 

Although our results provide evidenee for a positive impaet on FSL on presehool ehildren’ s 
understanding of seience and seientifie thinking, they are also mixed — a common situation in 
educational research. In this paper, we highlight a promising approaeh to learning from mixed 
(summative) results — by using formative assessment to provide more nuaneed information about 
children’s learning over time and to allow teaehers to tailor instruetion so that it is more 
effeetive. 
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Figure 1. Example of Instruction-Embedded Formative Assessment {Evidence Form for Open Exploration of Structures) 


Session 2: Open Exploration of 
Structures 

Evidence Form 

Date: 

Participating Children: 


1 ) 

2 ) 

3) 

4) 





Goals 

Indicators 

o Explores varied building materials, 
experiencing their differing 
characteristics and finding out what they 
can do with them 

o Develop purposeful engagement with 

• Uses a variety of blocks 

• Engages with blocks for extended periods 

• Attempts to solve problems that arise 

building materials from which interests 

• Shares ideas about the materials and 

and questions emerge 

process of building 

o Build capacity to participate in 

• Uses some descriptive vocabulary for 

conversations about materials and 

explaining block characteristics and 

building, generating a shared vocabulary 
about structures 

comparing similarities and differences 
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Please record your observations of children's behavior and conversation. When appropriate, attach children's representations. 


Child Behavior 

Child in Conversation 

OPrompt: We're going to spend time building with blocks today. How would you like to use the blocks? 



^Prompt: Here are two different kinds of blocks (show blocks of different shapes and materials). How are they different? 



O Prompt; Tell me about what you built today? Which blocks did you use? 
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Q Observe: Describe how children respond during block play, noting types of structures built and blocks used. Choose 2-3 children and time 
how long they are actively engaged in the block area. 
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Session 2: Open Exploration of Structures 


Do you have evidence of meeting your goals? 

Looking at the evidence you collected on the Evidence Form, indicate how well each Learning & Inquiry Goal was met, citing 
Evidence Item numbers (O'©) Evidence Formats: Child Behavior (B), Child Representation (R) or Child in Conversation (C). For 
Learning & Inquiry Goals marked or v^, indicate Next Steps you will take to help children reach these goals. 


Indicators 

Examples 

Evidence of Meeting Goals? 

, v^, v^+; Evidence Item #s & Formats) 

Next Steps 

• Uses a variety of 
blocks 

Child builds a structure using both 
kapla blocks and unit blocks; 

Child uses different blocks than 
he/she had been using previous day 



• Engages with 
blocks for 
extended periods 

Child is focused on blocks for 20-30 
minutes; 

Child chooses to stay in block area 
for duration of choice time 



• Attempts to solve 
problems that arise 

Child notices that a tower of soft 
blocks keeps falling over and tries to 
use other blocks 



• Shares ideas about 
the materials and 
process of building 

Child converses with peer about 
where to place blocks as they build a 
structure 

Child responds to teacher question 
about why he/she chose a certain 
block 
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